Recognizing text in images is useful in many computer vision applications such as image search, document analysis, and robot navigation. The ocr function provides an easy way to add text recognition functionality to a wide range of applications.

The ocr functions returns the recognized text, the recognition confidence, and the location of the text in the original image. You can use this information to identify the location of misclassified text within the image.

Here, the logo in the business card is incorrectly classified as a text character. These kind of OCR errors can be identified using the confidence values before any further processing takes place.

ocr performs best when the text is located on a uniform background and is formatted like a document. When the text appears on a non-uniform background, additional pre-processing steps are required to get the best OCR results. In this part of the example, you will try to locate the digits on a keypad. Although, the keypad image may appear to be easy for OCR, it is actually quite challenging because the text is on a non-uniform background.

The empty results.Text indicates that no text is recognized. In the keypad image, the text is sparse and located on an irregular background. In this case, the heuristics used for document layout analysis within ocr might be failing to find blocks of text within the image, and, as a result, text recognition fails. In this situation, disabling the automatic layout analysis, using the 'TextLayout' parameter, may help improve the results.

What Went Wrong?

Adjusting the 'TextLayout' parameter did not help. To understand why OCR continues to fail, you have to investigate the initial binarization step performed within ocr. You can use imbinarize to check this initial binarization step because both ocr and the default 'global' method in imbinarize use Otsu's method for image binarization.

After thresholding, the binary image contains no text. This is why ocr failed to recognize any text in the original image. You can help improve the results by pre-processing the image to improve text segmentation. The next part of the example explores two useful pre-processing techniques.

The poor text segmentation seen above is caused by the non-uniform background in the image, i.e. the light-gray keys surrounded by dark gray. You can use the following pre-processing technique to remove the background variations and improve the text segmentation. Additional details about this technique are given in the example entitled Correct Nonuniform Illumination and Analyze Foreground Objects.

After removing the background variation, the digits are now visible in the binary image. However, there are a few artifacts at the edge of the keys and the small text next to the digits that may continue to hinder accurate OCR of the whole image. Additional pre-processing using morphological reconstruction helps to remove these artifacts and produce a cleaner image for OCR.

After these pre-processing steps, the digits are now well segmented from the background and ocr produces better results.

There is some "noise" in the results due to the smaller text next to the digits. Also, the digit 0, is falsely recognized as the letter 'o'. This type of error may happen when two characters have similar shapes and there is not enough surrounding text for the ocr function to determine the best classification for a specific character. Despite the "noisy" results, you can still find the digit locations in the original image using the locateText method with the OCR results.

The locateText method supports regular expressions so you can ignore irrelevant text.

Another approach to improve the results is to leverage a priori knowledge about the text within the image. In this example, the text you are interested in contains only numeric digits. You can improve the results by constraining ocr to only select the best matches from the set '0123456789'.

The results now only have characters from the digit character set. However, you can see that several non-digit characters in the image are falsely recognized as digits. This can happen when a non-digit character closely resembles one of the digits.

You can use the fact that there are only 10 digits on the keypad along with the character confidences to find the 10 best digits.

In some situations, just pre-processing the image may not be sufficient to achieve good OCR results. One approach to use in this situation, is to identify specific regions in the image that ocr should process. In the keypad example image, these regions would be those that just contain the digits. You may select the regions manually using imrect, or you can automate the process. For information about how to automatically detect text regions, see Automatically Detect and Recognize Text Using MSER and OCR and Automatically Detect and Recognize Text Using Pretrained CRAFT Network and OCR. In this example, you will use vision.BlobAnalysis to find the digits on the keypad.

There are many connected regions within the keypad image. Small regions are not likely to contain any text and can be removed using the area statistic returned by vision.BlobAnalysis. Here, regions having an area smaller than 300 are removed.

Further processing based on a region's aspect ratio is applied to identify regions that are likely to contain a single character. This helps to remove the smaller text characters that are jumbled together next to the digits. In general, the larger the text the easier it is for ocr to recognize.

The remaining regions can be passed into the ocr function, which accepts rectangular regions of interest as input. The size of the regions are increased slightly to include additional background pixels around the text characters. This helps to improve the internal heuristics used to determine the polarity of the text on the background (e.g. light text on a dark background vs. dark text on a light background).

The recognized text can be displayed on the original image using insertObjectAnnotation. The deblank function is used to remove any trailing characters, such as white space or new lines. There are a few missing classifications in these results (e.g. the digit 8) that are correctable using additional pre-processing techniques.

Although vision.BlobAnalysis enabled you to find the digits in the keypad image, it may not work as well for images of natural scenes where there are many objects in addition to the text. For these types of images, the technique shown in the example Automatically Detect and Recognize Text Using Pretrained CRAFT Network and OCR may provide better text detection results.

This example showed how the ocr function can be used to recognize text in images, and how a seemingly easy image for OCR required extra pre-processing steps to produce good results.